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3 FUNCTIONAL ELEMENT TEST TOOL AND METHOD 

4 

5 STATEMENT OF THE GOVERNMENT INTEREST 

6 The invention described herein may be manufactured and used 



7 by or for the Government of the United States of America for 
i§ Governmental purposes without the payment of any royalties 
!!§ thereon or therefore . 



g CROSS-REFERENCE TO RELATED PATENT APPLICATIONS 

iEf Not applicable. 

O 

B BACKGROUND OF THE INVENTION 

15 (1) Field of the Invention 

16 The present invention relates generally to software testing 

17 and, more specifically, to a software test tool operable for 

18 providing an independent test environment for testing computer 

19 program system functional elements which may have multiple 

20 Application Data Exchange (ADE) interfaces. 
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1 Description of the Prior Art 

2 Large scale hardware and software systems and application 

3 may typically include many different functional elements or 

4 modules. In many cases, the various functional elements or 

5 modules are developed concurrently. As used herein, a 

6 functional element is a software module which performs a unique 

7 software task and which may have multiple interfaces with other 

8 functional elements and/or with an application comprised of 
numerous functional elements and/or with an overall system 

W comprised of a plurality of applications. In a preferred 

W embodiment of the present invention, the functional element 

lj2 performs one or more tasks which utilize an inter-task interface 

Itf or module-to-module communication protocol or mechanism. Each 

13 functional element may have multiple interfaces. The interface 

f5 sets forth constraints on formats, timing, and/or other factors 

16 required by an interaction of functional elements that perform 

17 different tasks within a computer system. 

18 Once the system is assembled, various problems may occur 

19 that cannot be easily traced to a particular functional element 

20 or module. In many cases, nhe problems relate to errors that 

21 occur at the functional element or module interfaces. As an 

22 example, one module or functional element might test and analyze 
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1 different underwater acoustic propagation signal propagation 

2 loss models. If the overall system response is not as expected, 

3 it may be difficult to determine whether the particular 

4 functional element for analyzing signal propagation loss is 

5 operating correctly or whether the fault lies elsewhere. 

6 Moreover, it may typically be difficult to monitor software 

7 interface operation of any particular functional element with 

8 respect to other functional elements and the overall system. 

Where systems involve complex meshings of inter-process 

W communications, a known technique for implementing the 

fi communication of task messages has been to use a shared memory 

lj2 for passing data between tasks. However, with this technique 

ffcf the problem of determining the fault which produces a given 

O error is even greater. 

JJ Consequently, there remains a long felt but unsolved need 

16 for improved testing of functional elements to ensure that 

17 accurate processing occurs within each functional element and 

18 that communications between the functional elements are 

19 consistent with the interface protocols. Those skilled in the 

20 art will appreciate the present invention that addresses the 

21 above and other problems. 
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1 SUMMARY OF THE INVENTION 

2 Accordingly, it is an object of the present invention to 

3 provide an improved system and method for testing functional 

4 elements of a computer software system. 

5 Another object is to provide a system and method as 

6 aforesaid which provides a controlled test environment that can 

7 facilitate off-line quantitative analysis of the test results 

8 and collected data. 

i § A further object is to provide a system and method as 

Hp aforesaid which is a stand alone functional test tool that is 

W much easier and faster to operate than a debugger or hand 

"(2 inspection of the application output results. 

ftf A still further object is to provide a system and method as 

Q aforesaid which enables software to reduce the overall number of 

Jft defects that occur during the development phase. 

16 A yet further object is to provide a system and method as 

17 aforesaid which is of special utility in application systems in 

18 which inter-process communication is implemented by techniques 

19 of using a shared memory scheme in order to pass data between 

20 tasks. 

21 These and other objects, features, and advantages of the 

22 present invention will become apparent from the drawings, the 
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1 descriptions given herein, and the appended claims. However, it 

2 will be understood that above listed objects and advantages of 

3 the invention are intended only as an aid in understanding 

4 aspects of the invention, are not intended to limit the 

5 invention in any way, and do not form a comprehensive list of 

6 objects, features, and advantages. 

7 In accordance with the present invention, a method is 

8 disclosed for a stand-alone testing environment for a functional 
element of an application. The application may be a subpart of 

19 an overall system. The functional element may have a plurality 

li of interfaces with the application and/or overall system. The 

¥2 method may comprise one or more steps such as, for instance, 

Q providing a computerized dialog to permit a user to create an 

M input data file for the functional element, prompting a user for 

f5 functional element interface tasks that have been previously 

16 developed utilizing the stand-alone testing environment. 

17 Alternatively, the dialog permits a user to start the functional 

18 interface tasks and a user supplied application system task. 

19 The plurality of interfaces are monitored. 

20 Other steps may include storing a unique interface file for 

21 the functional element interface tasks and/or displaying a 

22 status window while running the functional interface tasks. In 
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1 a preferred embodiment, the step of creating a test case 

2 generation file provides the user with task creation options 

3 related to the functional interface tasks. Other steps may 

4 include compiling the test case generation file to create a test 

5 case executable file for producing the functional interface 

6 tasks and/or storing the input data file in a user defined file 

7 such that the user defined file may be viewed and edited outside 

8 of the stand alone testing environment. 

An embodiment of the present invention provides a method 

MS for testing a functional element of a computer system with a 

fi stand-alone functional element test tool wherein the functional 

12 element has at least one interface for communicating with other 

f§ functional elements of the computer system using an interface 

Q protocol. In this embodiment, the method comprises one or more 

f5 steps such as, for instance, creating an input data file for the 

16 functional element by prompting a user for data format and 

17 content, storing the input data file, creating a test generation 

18 file by providing the user with a plurality of task creation 

19 options such that the selected task creation options are input 

20 into the test generation file, compiling the test generation 

21 file to produce a test case executable file in a predetermined 

22 inter-process communication protocol based on the selected task 
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1 creation options, initiating a test utilizing the test case 

2 executable file and the input data file for testing the 

3 functional element and the interface (s) by monitoring a status 

4 of the test, and storing test result data related to the test. 

5 The step of creating a test generation file may further 

6 comprise selecting test initiation features and/or providing at 

7 least one user defined button wherein the user defined button is 

8 user operable for the step of initiating the test. The method 
|J may further comprise playing back the test result data and/or 
1)0 providing a file viewer for the input data. Thus, the method 
ft permits comparing the test result data with expected results 

12 from the functional element utilizing the input data file. 

13 Thus, the present invention also provides a system 

14 operative for testing performance validity and accuracy of a 

Jj first computer program functional element which may comprise one 

16 or more elements such as, for instance, a test case data file 

17 producing subsystem for facilitating the production by a user of 

18 at least one file of test case data. The test case data 

19 producing subsystem can then be used for identification of an 

20 input data structure in order to prompt a user for input values 

21 of the test case data. Preferably, the test case data producing 

22 subsystem is operative to store one or more files of test case 
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1 data. Other elements may include a test case generation file 

2 producing subsystem for facilitating the production by a user of 

3 a test case generation file. The test case generation file 

4 producing subsystem provides a plurality of user options to 

5 facilitate the user in testing operation of the first computer 

6 program functional element and at least one communication 

7 interface. Moreover, the system may comprise a test case 

8 execution subsystem to operate the first functional element 

t S} based on the selected user options and the test case data. The 

(Up test case execution subsystem preferably provides a monitor for 

M operation of the first functional element. Preferably the test 

: Q case execution subsystem is operable for executing operation of 

jB a compiled executable file produced from the test case 

CI generation file. The test case execution subsystem may 

jft preferably be operable to effect operation of a second 

16 functional element simultaneously with operation of the first 

17 functional element such that the test case execution subsystem 

18 is operable to monitor the interface between the first function 

19 element and the second functional element. The test case 

20 execution producing subsystem is also preferably operative to 

21 tag an output of the first functional element with an indication 

22 of the task status. The test case generation file producing 
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1 subsystem is also preferably operative to facilitate the user 

2 selecting a predetermined test initiation event to start flow of 

3 the test case data into the first functional element. 
4 

5 BRIEF DESCRIPTION OF THE DRAWINGS 

6 A more complete understanding of the invention and many of 

7 the attendant advantages thereto will be readily appreciated as 

8 the same becomes better understood by reference to the following 
l§ detailed description when considered in conjunction with the 

IH accompanying drawings wherein corresponding reference characters 

\4 indicate corresponding parts throughout several views of the 

Q drawings and wherein: 

13 FIG. 1 is a schematic which indicates the general 

B functioning of a stand-alone test tool in accord with the 

K pre sent invent ion ; 

16 FIG. 2 is a schematic of an embodiment of a stand-alone 

17 test tool as the tool may be configured to cooperate with a 

18 given computer application system; and 

19 FIG. 3 is a block diagram indicating a flow and structure 

20 of a stand-alone test tool in accord with the present invention. 
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1 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

2 Referring to FIG. 1, in accord with the present invention, 

3 a stand-alone functional element (SAFE) test tool 10 has been 

4 developed to provide an independent test environment for one or 

5 more functional elements 18, which are desired to be the object 

6 of testing, within a given application or system. In a 

7 preferred embodiment, the SAFE test tool can be utilized to test 

8 any test object software which utilizes an inter-module 
communication mechanism or interface protocol. 

W An illustrative embodiment of the inter-process 

13 communication protocol is the Application Data Exchange (ADE) 
lj2 protocol, which has been custom developed for the Navy for use 
jlj in connection with software application programs. Illustrative 
S of one such Navy application program is a Sensor Performance 

14 Prediction Functional Segment (SPPFS) large-scale software 

16 module. SPPFS has multiple interfaces with software programs of 

17 an integrated system of ASW equipment digital processors. 

18 Briefly, SPPFS performs casting and analysis of different 

19 acoustic signal propagation loss models used in ASW combat 

20 control. More particularly, it functions in connection with 

21 sonar-based antisubmarine warfare equipment known as system 

22 AN/SQQ-89(V) 15. However, both the SPPFS and the AN/SQQ-89 (V) 15 
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1 are referred to for purposes of illustrating an environment, in 

2 which the present invention is utilized, and in and of 

3 themselves form no part of the invention. 

4 ADE uses queues to communicate between application tasks. 

5 The concept of queue within ADE can be viewed as a mechanism for 

6 inter-process signaling. Each queue consists of one or more 

7 messages. Messages in turn contain information. An example of 

8 a queue with an implementation in the environment of ASW systems 
^ is the queue which is used by a propagation loss analysis 

1§ function task element of a sonar signal analysis computer 

13 program to send equipment settings to the program's interface 

Ij function task elements. This provides communication between the 
two tasks. 

H ADE uses a shared memory scheme in order to pass data 

B between tasks. ADE manages the shared memory. Therefore when 

16 employed with SPPFS this functionality is invisible to the 

17 implementation of the SPPFS program task. However, the memory 

18 requirements of the implementation must be known and allocated 

19 via ADE functions (e.g., AdeAlloc) . Message passing with ADE 

20 can be viewed as the notification by an ADE protocol message to 

21 a task that information (sometimes in this specification and its 
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appended claims simply referred to as "data") is ready and that 
the data is located at a specific shared memory location. 

ADE uses a hierarchical structure for storing information 
within the ADE database. There are four levels within the 
hierarchy, w Queue " , "Application", "Topic", and "Item", where 
Queue is the most general and Item is the most specific. A Queue 
represents a highest level within the hierarchy. Within the 
SPPFS application program implementation, the Queue is typically 
the interface with the high level task. The Application is the 
second level and is associated with a queue. The functionality 
of an Application is typically more specific than that of a 
queue. However, within the SPPFS application the two often 
encompass the same functionality. The third level is a Topic 
which is associated with an Application. Again, the Topic is 
more specific than the queue. Finally the fourth level is the 
Item which is based on a Topic. 

The SAFE test tool provides an excellent test bed for 
testing functional element changes as well as software drops 
because the user has the capability of generating, collecting, 
and responding to all interfaces of the specified system 
functional element. Although the SAFE test tool prompts the 
user for input to perform testing, it is assumed that the user 
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1 understands the test object functional element operation within 

2 the overall system as well as the inter-module communication or 

3 interface protocol, e.g., the ADE protocol. 

4 Referring again to FIG. 1, there is schematically shown 

5 SAFE test tool 10 interconnected with any number of interfaces, 

6 as indicated by 12, 14, and 16, of test object functional 

7 element 18 to provide an independent test environment. Each of 

8 interfaces 12, 14, and 16 may have several modes of operation 
^ which may be tested in accord with the present invention. SAFE 
jj§ test tool 10 provides a stand-alone test environment that may be 
Kt utilized to simulate the application or overall system 19 with 
!;§ respect to test object functional element 18 and monitor any 

y number of interfaces for proper operation, such as element 

fcf interfaces 12, 14, and 16. The SAFE test tool 10 and the 

0 application or system 19 each communicate via a suitable linking 

16 arrangement, such as computer bus 19-1, with the above -discussed 

17 shared memory 19-2, that is involved in the shared memory scheme 

18 to pass data between tasks. Also per this earlier discussion, 

19 shared memory 19-2 has four levels of hierarchical structure: 

20 Queue 19-2A, Application 19-2B, Topic 19-2C, and Item 19-2D. 

21 SAFE test tool 10 can also be utilized in other 

22 configurations. For instance, in FIG. 2, SAFE test tool 10 may 
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1 be connected to one or more test object functional elements, 

2 such as functional elements 1, functional element 2, functional 

3 element 3, and functional element N, designated as 20, 22, 23, 

4 and 24, respectively, wherein communication paths 26, 28, 29, 

5 and 30, may comprise one or more ADE interfaces. Each ADE 

6 interface may have several modes of operation. In this 

7 embodiment, the test object functional elements may be part of 

8 or form a software application program such as, for example, one 
|i? of the functional elements of the above described Sensor 

£§ Performance Prediction Functional Segment (SPPFS) 31. In turn, 

jH SPPFS is part of an overall large-scale integrated system, such 

g as the above referred to AN/SQQ-89 (V) 15 combat control system 

fcf (not shown) . The functional elements of SPPFS interface with 

O various application programs of the large scale integrated 

if system through multiple interfaces, with interface tasks 

16 constituting inter-process communications. There may be other 

17 elements such as a CORBA (common object request broker 

18 architecture) link 31-1 and a CORBA software bridge 32 which may 

19 be utilized with many functional elements utilizing, preferably, 

20 ADE interfaces. In another arrangement, AN/SQQ-89 (V) 15 External 

21 Interfaces 34 as indicated in dashed lines, may operate through 
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1 link 31-1 and CORBA bridge 32 for testing one or more functional 

2 elements 20-24 and the various associated interfaces. 

3 FIG. 3 provides an overview of the details of operation of 

4 SAFE test tool 10. In order to effectively interface with a 

5 test object functional element, i.e., the functional element 

6 involved in the run test case step 86 of a test. SAFE test tool 

7 10 preferably operates in several selectable modes of operation 

8 or, in other words, is comprised of several selectable 
;| subsystems, as indicated at select subsystem 50. These 

il subsystems of SAFE test tool 10 permit the user to create input 

M data 52, create an interface test 54, application, and to run 

ife the tests 56. 

Qj Input file creator mode or subsystem 52 of SAFE test tool 

id 10 provides a mechanism that allows a user to create one or more 

B input data files that can eventually be accessed during test run 

16 time. During input file creation, SAFE test tool 10 provides a 

17 dialog with the user as indicated at 58 and 60. SAFE test tool 

18 10 allows the user to identify the structure of the input data 

19 and then uses this structure to prompt the user as to possible 

20 input values 60 in conjunction with user dialog 58. This is 

21 accomplished by prompting the user for data format and content. 

22 The format may be manually entered or the format may be 
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1 automatically determined. For instance, utilizing a drop-down 

2 form or other selection means, the user may identify the data 

3 structure to be utilized. Once the data format is identified, 

4 the data format is then utilized by SAFE test tool 10 to prompt 

5 the user to enter input values. When completed, the data is 

6 stored in a user-defined data input file 62. User-defined data 

7 input file 62 can now be used as an input file for future test 

8 cases. Preferably, the format of user-defined data input file 
*J| 62 is such that the file contents may be viewed and modified 

|| outside of the environment of SAFE test tool 10 for ease of use. 

14 For instance, the file may be readable utilizing file editor 64 

fi which may be a text file editor or any other suitable editor 

W which preferably does not require the use of SAFE test tool 10. 
fif Create interface test case mode or subsystem 54 of SAFE 10 

t§ provides a mechanism that allows the user to create a test case 

16 generation file. In one illustrative embodiment, the test 

17 generation file is recorded in the form of a high-level 

18 interface language, custom developed for ADE. The development 

19 of such a custom language is within the routine skill of 

20 programmers who develop special purpose compilers. More 

21 generally, the high-level interface language can be a 

22 development for any other interface protocol where such other 
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interface protocol is involved. In this embodiment, it is 
assumed that the user is knowledgeable of the functional element 
interface task to be tested. The file generation is 
accomplished by providing the user with interface task creation 
options 66 and user input 68 and may include options such as 
Create Queue, Edit Queue, Begin Enumeration, and the like. 

This description now proceeds to details of various 
preferred functional element task creation options. Create 
Queue allows SAFE test tool 10 to be able to create a Queue 
component of an ADE message which represents a function or 
functions to be performed for a test case. This component 
comprises the information (herein and the appended claims 
sometimes simply referred to as "data") stored in the Queue 
level of the ADE hierarchical shared memory. For each Queue 
message component, SAFE test tool 10 allows the creation of a 
user-defined Queue message component processing procedure or the 
use of a default Queue message component processing procedure. 
SAFE test tool 10 preferably has an edit queue option whereby 
SAFE test tool 10 allows editing of each queue. However, in a 
one preferred embodiment, once the Queue is created, SAFE test 
tool 10 is not able to delete the Queue message component. 
Another option is the Process Queue Messages option whereby SAFE 

18 



1 test tool 10 preferably is able to add messages that will be 

2 automatically handled within the test case. For this purpose, 

3 SAFE test tool 10 writes data to an output file specified by the 

4 user. SAFE test tool 10 allows for the processing of multiple 

5 message contents in the same Queue message component. SAFE test 

6 tool 10 does not allow the addition of messages until the Queue 

7 message component for the test case has been created. The 

8 option is also provided for creating an Application component of 
^ an ADE message comprising data stored in the Application level 
|§ of the shared memory, whereby SAFE test tool 10 preferably 

IB permits creation of multiple Application message components for 

ij2 each test queue. SAFE test tool 10 permits Application message 

jU| component creation only after a Queue message component has been 

H successfully created. An Edit Application option permits SAFE 

13 test tool 10 to modify each Application message component. A 

16 Create Topic message component option permits SAFE tool 10 to 

17 similarly create multiple Topic message components for each test 

18 application. Topic message components are capable of being 

19 created only after an Application message component has been 

20 successfully created. Edit Topic permits modification of each 

21 Topic message component. Create/Edit Item allows SAFE test tool 

22 10 to be able to create and/or edit multiple Items message 
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1 components. Note that Items message components are created only 

2 after a Topic message component has been successfully created. 

3 Enumerate on Application, Topic, or Queue, allows SAFE test tool 

4 10 to be able to begin an enumeration on an application, topic, 

5 or item that has been defined in the task to be tested. SAFE 

6 test tool 10 automatically saves the server, Queue message 

7 component, and Topic message component identifiers upon 

8 successful enumeration. In addition, SAFE test tool 10 
preferably provides the capability of allocating memory, 

§§ requesting data, or requesting notice when the enumeration is 

13 satisfied. An Allocate/Free Memory option permits SAFE test 

i|2 tool 10 to be able to allocate and free memory and associate 

W each memory block action with a given action (e.g., 

S enumeration) . A Send/Receive Data option permits SAFE test tool 

13 10 to send and receive data to the task being tested. An 

16 Initiate Input Data option preferably permits SAFE test tool 10 

17 to provide the capability of initiating input data by allowing 

18 the user to create, for instance, buttons or other indicators. 

19 Upon selection of the button, SAFE test tool 10 would then 

20 initiate data to the interface and functional element being 

21 tested. The input data will be accessed from a user defined 

22 input data file. The option to Insert Header Files permits SAFE 
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1 test tool 10 to provide the user with the ability to insert 

2 header files into the software as needed. This can preferably 

3 be performed automatically or manually. 

4 The selected options are input into a generation file that 

5 is used to create the functional element test case. One of the 

6 options, as discussed above, permits the user to accept/initiate 

7 data from/to the function element for the interface (preferably 

8 ADE) task being tested. Data may preferably be initiated 

: 9 automatically upon the occurrence of some event (e.g., receipt 

||> of a message) or manually via a user-defined button as discussed 

IB above. Upon completion of the test case generation process, 

If test case generation file 70 is available. In a preferred 

embodiment, test case generation file 70 is compiled outside of 

14 SAFE test tool 10 utilizing, for instance, by a compiler 72 

Q which compiles the various options selected to create test case 

16 executable 74 . 

17 Test case execution 56 of SAFE test tool 10 provides a 

18 mechanism to run multiple test case executables when testing a 

19 functional element application task as indicated at 84 and 86. 

20 In a preferred embodiment, this testing employs a predetermined 

21 interface protocol for inter-process communications, e.g., the 

22 ADE protocol. Test case executable files 74 are therefore the 
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compiled executable version of the test case generation file 
created within the test case creation 54 mode. For each test 
case, SAFE test tool 10 preferably provides a monitoring window, 
as indicated at 78, that displays the current status of the 
given task. During test case generation stage 54, preferably 
print statements (for instance print statements) are 
automatically inserted that generate or print the status in 
monitoring window 78. In addition, the user define options 
(e.g., data initiation buttons) are displayed as dictated by 
each test case. The run test case step 86 is then performed 
under initiation by a start options function 84, using as inputs 
the user defined filed 62 and test case executable 74. Upon 
completion of the task execution, the results are output to the 
test case and may be stored in data storage 8 0 for future 
analysis or playback such as indicated at 82. SAFE test tool 10 
is operable to maintain a unique interface file for each 
functional element task as prescribed by the user. 

Reference is again made to FIG. 3, for an example of use of 
SAFE test tool 10 in connection with the SPPFS software 
application program (see reference character 31, FIG. 2) and the 
operation of SPFFS in examining and analyzing different acoustic 
signal propagation loss models. Further, in this example, it is 
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1 assumed that SAFE test tool 10 employs the ADE interface 

2 protocol for inter-process communications. Both the interface 

3 requirements and the specific input data requirements are well 

4 defined within the SPPFS application. Varying input data can be 

5 supplied and outputs can be collected to data files for off-line 

6 quantitative analysis utilizing playback 82, e.g., pointwise 

7 comparison with simulated/expected results. In addition, 

8 comparison of the different acoustic signal propagation models 
^ with different environments can be facilitated via SAFE test 
g> tool 10. When using SAFE test tool 10 in this way, the user 
tfl would be prompted for the command line that initiates the 

t| selected SPPFS functional element to be the object of testing by 

y SAFE test tool 10. In addition, the user would be prompted for 

the interface tasks as developed with interface task creation 

0 options 66. These tasks would be initiated via SAFE test tool 

16 10 with test case executable 74. Upon successful initiation at 

17 84, the user would have the ability to initiate inputs to the 

18 functional element to be tested. The responses from the test 

19 object functional element being tested during run test case step 

20 86 would be maintained via SAFE test tool 10 for later 

21 comparison or output validation. 
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1 In accordance with the present invention, any application 

2 that utilizes a particular inter-module communication mechanism 

3 or interface such as the ADE protocol, or other interface 

4 protocols, will be significantly enhanced by the use an 

5 embodiment of SAFE test tool 10. SAFE test tool 10 facilitates 

6 functional level testing and evaluation of the application 

7 system (in the example, SPPFS) functional elements independently 

8 of the remainder of the system. In addition, SAFE test tool 10 
jJJ provides a controlled test environment that can facilitate off- 
ffl line quantitative analysis of the test results and collected 

M data. Finally, SAFE test tool 10 provides a good environment 

l| for verifying software changes and status at a functional 

|J element level. This mechanism is therefore a valuable tool in 

H the validation of software performance and processing accuracy, 

p Quality software can be developed in less time and with fewer 

16 overall defects due to use of SAFE test tool 10 during the 

17 development phase. This affords significant cost savings both 

18 during the initial development phase as well as overall life 

19 cycle maintenance. 

20 It will be appreciated by those skilled in the art that the 

21 invention can be implemented using a suitable programmed general 

22 purpose computer or special purpose hardware, with program 
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1 routines or logical circuit sets performing as processors. Such 

2 routines or logical circuit sets may also be referred to as 

3 processors or the like. 

4 Therefore, it will be understood that many additional 

5 changes in the details, materials, steps and arrangement of 

6 parts, which have been herein described and illustrated in order 

7 to explain the nature of the invention, may be made by those 

8 skilled in the art within the principle and scope of the 

9 invention as expressed in the appended claims . 
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